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POTENTIAL TARGET IDENTIFICATION 

It is advantageous to detect or identify image elements or targets as far 
away as possible. For example, in battle situations, candidate or potential 
5 targets should be detected early, increasing the likelihood of an early detection 
of a target or other object. For a simple background scene such as a blue sky, 
a target may be recognized from a relatively long range distance. However, for 
some high clutter situations such as mountains and cities, the detection range 
is severely reduced. Moreover, such clutter situation are often complicated to 
10 process. For example, the background may be mixed with different clutter 
types and groups. Also the background clutter may be non-stationary. In 
these types of situations, the traditional constant false alarm ratio (CFAR) 
detection technique often fails. 

To solve the problems discussed above, spatio-temporal fusion 
15 techniques is proposed for enhancing target detection (recognition). Several 
advanced detection schemes are proposed as well including a double- 
thresholding technique and a reverse-thresholing technique. These techniques 
may be applied to the complicated clutter situations where the traditional 

CFAR technique fails. 
20 Also, pre-detection temporal integration and spatial fusion techniques 

are proposed for enhancing target detection and recognition. These techniques 
involve different spatio-temporal fusion strategies such as the additive, 
multiplicative, MAX, and MIN fusions. In spatial fusion, extracted features 
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from different sensors are fused. In temporal fusion, extracted features across 
a multiple time frame window are fused and integrated. 

In addition, a double- thresholding technique is proposed when the 
background scene mixed with different clutter sub-groups. Some of them may 
5 have means larger than the target mean, while some of them may have means 
smaller than the target mean. This technique selects a lower bound threshold 
(below the target mean) and a higher bound threshold (above the target mean). 
This technique in combination with the spatio-temporal fusion techniques will 
threshold out most of the different clutter groups. 
10 Further, a reverse-thresholding technique is proposed when the 

background scene contains non-stationary clutters with increasing or 
decreasing means. The detection assignment criteria may be reversed 
depending on if the clutter mean is larger or smaller than the target mean. 

The sensor clutter noise looking at real scenes (trees, grass, roads, and 
15 buildings, etc.) has been studied. The sensor clutter noise at most (> 95%) of 
the sensor pixels is near stationary and un-correlated between pixels as well as 
(almost) un-correlated across time frames. The noise at a few pixels (looking at 
the grass near the road edge) has shown non-stationary properties (with 
increasing or decreasing mean across time). 
20 If clutters with broader pdf (probability density function) than the target 

are encountered, it may be important to further investigate if the broad clutter 
pdf is caused by non-stationary noise with a time-variant mean or is caused by 
a mix of different clutter types with different stationary means. Then different 
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detection techniques, such as the newly proposed double-thresholding or 
reverse-thresholding schemes, may be selected accordingly. 
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1. INTRODUCTION 

Spatio-temporal fusion for target classification is discussed. The fusion is conducted in the likelihood function 
reading domain. In general, the likelihood functions (pdfs) are obtained from training data based on single sensor and 
single frame measurements. Therefore, if we conducted fusion using the likelihood readings of the features extracted 
from measurements of single sensor and frame, we only need to store one set of hkehhood functions for single sensor 
and frame, no matter how many sensors and frames we will use for fusion. On the other hand, since the detection .process 
uses thresholding technique instead of likelihood functions, we can directly fuse the features values from different 
sensors and time frames in the feature domain for target detection. 

Spatial fusion is defined as the fusion between different sensors, and temporal fusion is defined as the temporal 
integration across different time frames within a single sensor. Different spatial fusion and temporal integration (fusion) 
strategies have been developed and compared, including pre-detection integration (such as additive^ multiplicative, 
MAX and MIN fusions), as well as the traditional post-detection integration (the persistency test). The pre-detection 
integration is conducted by fusing the feature values from different time frames before the thresholding process (the 
detection process), while the post-detection integration is conducted after the thresholding process. 

Although our techniques are aimed for improving target detection, these techniques can be used for other 
applications involving thresholding techniques. In target recognition, ATR (automatic target recognition) is a research 
area with high attention. One popular ATR approach uses the matched filtering/correlation techniques, and the resulting 
features after the correlation (e.g., the peak-to-sidelobc-ratio) will subject a threshold-screening to pick the recognized 
taigets. Therefore, both the pre- and post-detection temporal integration methods can be used to enhance target 
recognition when multiple temporal frames are involved. ... - 

Temporal correlation and non-stationary properties of sensor noise have been investigated using sequences of 
imaeery collected by an IR (256x256) sensor looking at different scenes (trees, grass, roads, buildings etc ). The natural 
noise extracted from the IR sensor, as well as noise generated by a computer with Gaussian and Rayleigh MM 
have been used to test and compare different temporal integration strategies. The simulation results show that both the 
pre- and post-detection temporal integrations can considerably enhance target detection by integrating only 3-5 time 
frames (tested by real sensor noise as well as computer generated noise). Moreover, the detection results can be further 
enhanced by combining both the pre- and post-detection temporal integrations. 

2. SPATIO-TEMPORAL FUSION 

For a physical sensor, the sensing errors are mainly caused by the measurement noise n a that is generally described 
as a RV (random variable)'. For target detection at the feature level 2 , different features are exacted from he onguial 
physical measurements. The extracted features affected by the measurement noise are also RVs. The pdf (probability 
density function) of a feature RV may or may not have the same distribution as the original measurement ^noise . To 
increase the Pd (probability of detection), we must reduce the influence of the feature RVs. The influence of RVs can be 
decreased by reducing the variances (<** ) of the RVs and/or by increasing the distance (J) between the means of the two 
feature RVs related to the target and the clutter). The reduced feature variances and/or the increased feature distances 
will increase the signal-to-clutler-noise-ratio (SCNR) and thus lead to a better ROC (receiver operating characteristics) 
performance, i.e., a higher Pd for the same Pfa (probability of false alarms). 



1 For example for an IR (infrared) sensor, the measurement noise (temporal noise) may be originated from scene-background, 
WhT uLmission, path radiance, optics, filters, sensor homing and shield, detector dark current, p.xel phasing, quantization, 
amnlifier and read-out electronics, etc. . . . . . 

MnT IR sensor, for detecting a resolved target occupying multiple pixels and or a unsolved target occupying only i ' ™^e targej * 
spatial matcM filtering prac«s in general is conducted before the detection (thresholding process. The filter can be a Sobel edge 
SoTa DOG (difference of GaJsian) filter, a specific tuned basis function, or an optical point spread fi.nct.on. The output of the 

^^T^^^Un and the exUacted feature is a tinear transform (e.g. the mean or average of 
ZSSSSL s "near feature) of the physical measurement, the distribution of the feature RV «ri 1 sti I taG_ On the 
other hand, if Ae relationship between the extracted feature and the original measurement is non-linear the feature d.stnbut.on u. 
ZTwi i be different from the original one. For example, for a ladar sensor with a Gaussian d.stributed measurement noise, f we 
us! rS'Iplhude oTthe radar return real and imaginary signals as the extracted feature, the distribution of the feature RV will be 
Rayleigh. 
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Figure 1. Spatio-temporal Fusion 

Two approaches for reducing the variance of RVs are 1) temporal integration between time frames by averaging the 
RVs m dSm fi^nes (the prelection integration), and 2) a binomial persistency test us.ng a wndow of tunc fames 
Ohe Jostltection integration). Wold in 1938 proposed and proved a theorem that gives us some ms.ght mto how 
temporal integration can be useful: 
Wold's Fundamental Theorem: 

Any stationary discrete-time stochastic process (x(n)} may be expressed in the form 

rtU2wl«% are uncorrelated process, u(n) is a RV. and s(n) a a deterministic process. 

Wore Jufn) is less temporally correlated, temporal integration will be more useful to reduce the variance 
of u(n) ? 5? ca'sl temporal integration^ multiple time frames (temporal fusion) can enhance detection and 

there is amtoer condition need to be addressed. In many realistic situations, the target may be moving and the sensor 
SoL^ mVvCreSed to the background clutters. Therefore, another critical condition for effective temporal 
Sn™ toa^ taking and associative targets and clutter objects (u,, the detected objects) at Afferent t me 
tones using navigation initial tracker and/or image-based tracker or any effect.ve .mage/object 
registration/association/correlation techniques. 

3. DIFFERENT FUSION STRATEGIES 

In this Section, we present four fusion (RV combination) strategies: 1) additive 4 , 2) multiplicative, 3) M1N, and 4) 
MAX fusion. 

3.1 Additive Fusion 

The additive fusion rule for two sensors (or two time frames) is 



4 Please refer to [1] and [2] 

sensors. 



for more detailed description of the additive fusion and its advantage when adaptively weighting different 
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p(t)=p(H)+pfi2), and p(c)=p(cl)+p(c2). (I) 

where p(t) is the fused target feature values, p(tl) and p(t2) are the target feature values at sensor 1 and sensor2 (or time 
framel and frame2), respectively; p(c) is the fused clutter feature values, p(cl) and p(c2) are the clutter feature values at 
sensor! and sensor2 (or time framel and frame2X respectively. In a frame, there are generally many more clutter feature 
values at different pixel locations. 

The additive fusion can be easily extended to include more than two sensors (spatial fusion) or more than two 
time frames (temporal integration): 

p(t) = p(tl) + p(t2) + +p(m), and p(c) =p(cl) + p(c2) + • + p(cn). (2) 

For two independent RVs: X and Y, the combined pdf of the summation of these two RVs (Z * X + Y) is 
calculated as the convolution of the two individual pdfs: 

f 2 (z) = )f x (x)f r (z-x)dx. (3) 

0 

In our additive fusion case (with two sensors or two frames), p(t) = z, p(tl) = x> andpft2) = y [or p(c) = z, pfcJ) 
= jt, and p(c2) - >]. From Eq. (3), we have 

/ P( ,)(/'(0) = ]/p(,,(^l))/ P( , 2) (/'(/)-/'(/l))*(rt) J (4) 

0 

and 



o 

Eqs. (4) and (5) can be used to predict the detection performance of the additive fusion, since the ROC curves after the 
additive fusion can be estimated from the combined pdfs in Eqs. (4) and (5). 

3.2 Multiplication Fusion 

The multiplicative fusion rule of two sensors (or two time frames) is 

p(t)=p(tl)*p(t2), and p(c)=p(cl)*p(c2). (6) 

For two independent RVs: X and Y, the combined pdf of the multiplication of these two RVs (Z = X * Y) is calculated 
as the nonlinear convolution (with divisions of a RV) of the two individual pdfs: 




In our two-sensor multiplication fusion case, from Eq. (7), we have 




and 



(9) 



3 J The Relationship Between Additive and Multiplication Fusions 
If we take the logarithm on both sides of the multiplication fusion equations [Eq. (6)], we have 

ln[p(t)]=ln[p(l!)] + ln[p(t2)], and ln[p(c)] = ln[p(cl)] + ln[p(c2)]. (10) 

The multiplication term becomes two additive terms oflogarithm functions in each of the equation. If we have 
two RVs with log-normal pdfs, the equations above indicate that the multiplicative fusion of two RVs with log-normal 
distributions is equivalent to the additive fusion of two RVs with normal distributions. 

3.4. MIN and MAX Fusions . . _ 

The conjunction (AND) and disjunction (OR) are two frequently used combination rules in Fuzzy Logic. For 
two independent RVs: X and Y, the combined pdf of the conjunction of these two RVs [Z - mm(X, Y)] is given as 



fz W = fx MP " F r «] + fy W " F * (z)1 ' 



(11) 



where F(z) is the cumulative distribution function. _ _ 

Similarly, for two independent RVs: X and Y, the combined pdr of the disjunction of these two RVs IZ - 

maxfX, Y)] is given as 

/ x (r)=AW(z)+/r(*V r x(*)- * (12) 

For our two-object problem, the MIN (conjunction) fusion is 

p(t)=min[p(tl),p(t2)], and p(c) = min[p(cl).p(c2)]. (13) 
The MAX (disjunction) fusion is 

p(l) = maxfpftl). p(t2)], and p(c) = maxfpfal). p(c2)]. (14) 

The terms of pre- and post-detection integrations were originally used in radar sensor detection. They can be equally 
applied for IR sensor detection. For both methods, a temporal moving integration widow (typically containing several 
frames ee N = 5 or 7) is first selected. In the pre-detection method, one of the different fusion strategies (discussed in 
the last" seition) is applied for the frames within the window size. The fused features values are then used for detection 
Applying thresholding). In the post-detection (also called persistency test) method, detechon (thresholding) is first 
perfoS on each inSge frame within the moving window (with N frames). Then evaluate how many detecnons out of 
the N frames occur for a detected object For example, for a criteria of 5-out-of-7, .f an object was detected from 5 or 
more frames in a moving window with 7 frames, the detected object is considered as a target. Otherwise, it is considered 
as noise or clutter detection. 

4. PRE- AND POST-DETECTION TEMPORAL INTEGRATION 

gllllil 




Figure 2. ROC Performances of Pre- and Post-Detection Integration 



Fig 2(a) shows the pdfs (probability density functions) for a noise and a target in a single frame with STD (standard 
deviation) = 5. Fig 2(b) shows the pdfs after averaging 25 frames (the pre-election integration: equivalent to the 
additive fusion). The accumulated probability curves (the erf functions) of the pdfs in Fig 2(a) and (b) are plotted m Fi£ 
2(c) (Blue curves: the single frame; Green curves: average of 25 frames). Several ROC curves are plotted m Fig 2(d) . 
The top and bottom (red) curves are 7-frame average and 3-frame average, respectively. The middle three (blue) curves 
are 4-out-of-7, 5-out-of-7, and 6-out-of-7 persistency test results. 
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Figure 3. Target Detection under a Two-target Situation 

Square: targetl; Circle: target2; Triangle: clutter noise 

5. SIMULATIONS OF TARGET DETECTION 

As shown in Fig 3, we simulated both IR and RF sensors for target detection enhancements using spatio-temporal 
fusion Spatial Fusion (integration) is conducted between the 1R and the RF frames (pre-detection integration only), 
while Temporal Fusion is conducted across several time frames of each sensor (both pre- and post-detection integration). 
Two target situations were simulated: 1) single-target in the scene, and 2) two-targets in the scene. In general, the single- 
target case has less adjustable parameters and thus would be easier to compare performances from different fusion 
strategies than the multiple-target case. However, the multiple-target case occurs at many realistic situations. A two- 
target case is shown in Fig 3. In this simulation, we used static targets and clutter, and presume perfect object tracking 
and/or registration across multiple time frames. 




iric m t 

Figure 4. The Gaussian pdfs of a Target and a Clutter Noise 

50 random data samples (related to 50 time frames) were generated as performance data set for each object (target or 
clutter noise) to evaluate the detection performance. The detection was conducted using the traditional CFAR (constant- 
false-alarm-ratio) strategy. For a specific CFAR threshold, each detected target at one of the 50 frames counts on 2% of 
Pd (probability of detection) for the single-target case, and 1% of Pd for the two-target case. The noise m IR is simulated 
as a normal distribution with a standard deviation of 10, and the noise in RF is simulated as a Rayleigh distribution with 



5 For the pre-dctection integration, the ROC curves are obtained by directly plotting the accumulated probability curves in Fig 2(c) 
the x- and y-axis in Fig 2(d). For a k-out-of-N post-deteoion integration, the accumulated probability curves need to be transferred 
post-detection accumulated probability curves using the binomial equation: p ^ k • jv ) = £ [ N J p * ( I - p)"'* 



a standard deviation of 6.5. Fig 4 shows the pdfs (probability density functions) of a target and a clutter noise with 
normal distribution. 

The detection ROC performances without any temporal integration (single frame) are shown in Fig 5 as a baseline 
performance to compare different temporal fusion strategies. Fig 5(a) shows the baseline result from an IR sensor, while 
Fig 5(b) shows that from a RF sensor. The y-axis is the Pd (probability of detection), and the x-axis is the false-alarm 
number per frame. The curve with circle symbols is the result from the single-target case, and the curve with square 
symbols is the result from the two-target case. It is seen that for a false alarm rate of two false alarms per frame the Pd is 
about 80% for IR and 85% for RF, and that the single-target case performs a little better that the two-target case. 
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For the four different fusion strategies discussed in Section 3, our simulation results for target detection show that 
the multiplication fusion performs the same as the additive fusion, and the MIN fusion performs a little better than the 
MAX fusion. In this paper, we show the results for the Additive and MrN fusion. 

The detection ROC performance curves for the single-target case of IR sensor are shown in Fig 6(a), while the 
detection ROC performance curves for the two-target case of IR sensor are shown in Fig 6(b). The curve with the circle 
symbols shows the baseline performance (single frame). The curve with the triangle symbols shows the result of spatial 
additive fusion between the IR and the RF sensor, while the curve with the square symbols shows the result of additive 
temporal fusion by integrating a time window of three frames. Similar results for the RF sensor are shown in Fig 6(c) 
and (d) It is found- 1) the spatial fusion enhance the detection than the single sensor along, although the IR (the worse 
sensor) enhanced more than the RF (the better sensor) did; 2) The temporal fusion using three time frames performs 
better that the spatial fusion using only two sensors. In general, if the noise in different frames are independent to each 
other a temporal fusion with N (=2,3,...) frames should perform similar to a spatial fusion with N sensors. We will 
discuss the noise correlation properties between frames in a later section. The results of additive temporal fusion using 
five time frames are shown in fig 7. It is seen that the target detection can be further enhanced by increasing the time 
window of integration. 

52 Additive Temporal Fusion vs. MIN Temporal Fusion 




The results comparing the additive fusion with the MIN fusion for an integration window of five frames are shown 
in Fig 8 Both additive and MIN fusions with multiple frames enhance target detection. For the IR sensor (with normal 
noise distribution), the additive fusion always outperforms the MIN fusion in both the single-target and two-target cases 
as shown in Fig 8(a) and (b), while for the RF sensor (with Rayleigh noise distribution), the MIN fusion can further 
enhance target detection, and performs equally well as the additive fusion in both the single-target and two-target cases 
as shown in Fig 8(c) and (d). 



Figure 9. Persistency Test (Window = 5 frames) 




Figure 10. Additive Fusion and Persistency Test (Window = 5 frames) 




Figure 11. Combination of Additive Fusion and Persistency Test 



5.3 Post-Detection Integration (Persistency Test) . 
The persistency test has been discussed and shown in Section 4 and Fig 2. Persistency test results for both IR and 

RF sensore are shown in Fig 9. The three curves in each figure are the persistency test for K out of N frames (K-2,3,4; 
and N=5). Similar to the result in Fig 2(d), the three curves in Fig 9 show similar detection enhancements. 

5.4 Additive Fusion vs. Persistency Test 

Figure 10 shows the results of additive fusion (the curve with square symbols) and the persistency test (the curve 
with triangle symbols) for both the IR and RF sensors. It is found from Fig 10 that by integrating only five frames, both 
additive fusion and persistency test can significantly enhance target detection from the baseline (single frame), with 
additive fusion performing a little better than the persistency test. 

Furthermore, the additive fusion and the persistency test can be complementary to each other. They can be 
combined to further enhance target detection. Results using an integration window of five frames are shown in Fig 1 . 
The curves with triangle symbols show the ROC performance of the persistency test, the curves with square symbols 
show the ROC performance of the additive fusion, and the curves with circle symbols show the combination ROC 
performance of the additive fusion and persistency test. 

6. TEMPORAL CORRELATION PROPERTIES OF REAL IR NOISE 

As discussed in the previous section, the performance of temporal integration depends on the temporal correlation 
properties of the sensor noise. The better performance can be achieved if the noise across the tune frames is less 
collated. In the simulate results presented in the previous section, we used computer generated random noise that is 
generally uncorrected between frames. What about the real sensor noise? To answer this question, we extracted and 
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studied die multiple frame noise from an InSb IR FPA (focal plane array) with 256x256 pixels. Imagery sequences (50 
time frames) were collected by this IR sensor looking at different scenes (trees, grass, roads, buildings, etc.). 



Figure 12. Auto-correlations of Real and Computer Generated Noise 
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Figure 13. Noise De-trend 

Studies of the natural IR noise have revealed that 1) the sensor noise at most (> 95%) of the sensor pixels are near 
stationary and un-conelated between pixels as well as (almost) un-correlated across time frames; and 2) the noise at a 
few pixels (e.g., the grass aside the road) has shown non-stationary properties (with increasing or decreasing mean 
across time) Fig 12(b) shows a typical stationary and uncorrelated noise sequence (50 frames) from a specific pixel. Its 
auto-correlation function is shown in Fig 12(a). Fig 12(d) shows a typical non-stationary noise sequence with a 
decreasing mean across time. Its autocorrelation function with high temporal correlation is shown in Fig 12(c). Fig 
12(e) shows the auto-correlation function of a Gaussian random noise sequence (50 frames) generated by a computer 
(this noise has been used in the simulation discussed in the previous section). It is seen that the natural noise and the 
computer-generated noise have similar autocorrelation functions [Fig 12(a) and (e)], and thus both are highly 

uncorrelated across time. . „ 

From the natural IR noise, we notice that the non-stationary noise at a specific pixel always shows high values off 
the center peak in the correlation function. To understand whether the high vales caused by the non-stationary properties 
only or caused by both non-stationary and temporal correlation, we have de-trended the non-stationary noise sequences, 
and remove the increasing or decreasing means. Then we found that the de-trended noise (becoming a stationary 
process) becomes temporally uncorrelated (low values off the center peak in the correlation function). This finding 
indicates that the noise at pixels with high off-center correlation values is non-stationaiy but not temporal correlated. 
One such example of the noise de-trend is shown in Fig 13. Fig 13(a) shows a non-stationary noise sequence with a 
increasing mean whose auto-correlation function is shown in Fig 13(b). Fig 13(c) shows the same noise after ^de-trend 
process, and its auto-coirelation function is shown in Fig 13(d). It is seen that the autocorrelation function in Fig 13(d) 
has much lower off-center-peak values than that in Fig 13(b). 
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We have applied the IR real noise to test our different temporal fusion strategies, as well as pre- and post-detection 
temporal integration. The performances using the stationary IR noise are similar to the performances using computer- 
generated noise as shown in the previous section. Fig 14(b) shows a stationary target noise sequence (50 frames, the 
solid curve) and a stationary clutter noise sequence (the dashed curve). The target detection ROC performances are 
shown in Fig 14(a). The curve with circle symbols shows the baseline (single frame) performance. The curve with 
triangle symbols shows the performance using persistency test with an integration window of 3 frames (2 out of 3), and 
the curve with square symbols shows the performance of additive fusion with an integration widow of 3 frames. Fig 
14(d) shows a non-stationary target noise sequence (the solid curve) with a decreasing mean and a stationary clutter 
noise sequence (the dashed curve). The target detection ROC performances are shown in Fig 14(c). It is seen that the 
detection performances are much worse than the results shown in Fig 14(a). 




Figure 14. Target Detection Using Real IR sensor Noise 
7. DISCUSSION AND SOME FURTHER THOUGHTS 
7.1 Temporal Fusion and IR Sensor Non-Uniformity Correction 

In the traditional NUC (non-uniformity correction) design, frame subtraction is generally used to subtract out the 
FPN (fixed pattern noise). However, direct subtraction of two adjacent frames will double the variance of the temporal 
noise To avoid a large increase of temporal noise, the NUC design is applied a feedback loop and only a small fraction 
of the FPN is subtracted out at each iteration. Nevertheless, if we apply temporal integration in the detection system after 
the NUC process, we can afford the direct subtraction between two nearby frames, and further reduce the noise. For 
example, the sum of n original frames results in a variance of nxv (where v is the single frame variance). On the other 
hand, the sum of n subtracted frames results in a variance of 2xv, because all the variances in the middle frames are 
cancelled out and only the two variances in the first and the last frames are leftover. Therefore, for an average of n 
original frames, the resulting variance is v/n, while averaging n subtracted frames, the resulting variance is 4v/rf. That 
is, (4v/n 2 ) < (v/n) when n > 4. 




Figure 15. Gaussian pdfs of Clutters and Target 

7.2 Double-Thresholding Detection Scheme . . 

If the feature values of all different clutters in a scene are lager (or smaller) than the target feature value as indicated 
in Fig 4 the traditional CFAR detection scheme will still works. For the example in Fig 4, the CFAR scheme always 
treats an' object with a feature value below the threshold as a clutter, and above the threshold as a target. However, in 
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reality, the chittcr situations are very complicated As shown in Fig 15, some clutter groups (e.g., some trees or roads) 
may have feature values lower that the target, while some other clutter groups (e.g., some decoy-like objects, or counter- 
measurement objects) may look more like the target and thus have feature values higher than the target. In these 
situations, the traditional CFAR scheme will partly fail because it only uses a single-thresholding scheme that can only 
threshold out one of the clutter group and left the other to be more likely false detected as a target. 

In the situation that some clutter feature values are larger than and some are smaller than the target feature value, we 
propose a double-thresholdim scheme with one up-bound threshold and one lower-bound threshold. The technique in 
combination with the temporal integration will considerably enhance target detection. For example, as shown in Fig 15, 
suppose the two clutters and the target have Gaussian distributions with the same variances. The separation of the target 
from the two clutters is two a (Le., two standard deviation): 

m t - m c i = m c2 " m t = 2<t 

If we set the double thresholds as one a below and one o above the target mean rn^the detection criteria is that only 
a object with a feature value larger than the lower bound threshold and smaller than the higher bound threshold is 
assigned as a detection. This is a two-sigma probability and for a Gaussian distribution the Pd (Probability of target 
detection) is around 68%, and the Pfa (probability of false-alarm) caused by the two clutter groups is around 34% (= 
17% + 17%). This is the baseline performance for the traditional single frame detection. However, if we apply the 
temporal integration of 9 frames 6 with the additive fusion (equivalent to averaging 9 frames), the standard deviations for 
the clutters and the target will be reduced by a factor of 3. Then this is a six-sigma probability. The Pd is increased to 
above 99%, and the Pfa caused by the two clutters is reduced to below 2%. 

In this technique, for appropriately selecting the two thresholds, we prefer to have die pre-knowledge of the target 
mean that may be available from some good training data. 

73 Reverse-Thresholding Detection Scheme 

Another situation that the traditional CFAR scheme will fail is when non-stationary targets and/or clutters exist. As 
shown in Fig 14(d) where a non-stationary target with a decreasing mean exists. At an earlier time moment, the target 
mean is larger than the clutter mean, while at a later time moment the target mean is below the clutter mean. For a 
traditional CFAR single-thresholding approach, we set a single threshold 7 , and any object with a feature value above it 
will be assigned as a detected target. This approach work at earlier time moments when the target mean is larger than the 
clutter mean. However, it will fail when the target mean moves close to and further below the clutter mean, the clutter 
will have much higher probability to be falsely detected as a target than the real target. That is why the detection 
performances in Fig 14(c) are much worse that those in Fig 14(a). 

Similarly, a non-stationary clutter situation can be easily understood using Fig 15. Suppose at an earlier moment the 
non-stationary clutter with a increasing mean was at the clutterl location. At a later time moment, it moved from the left- 
side of the target to the right-side of the target at the clutter2 location. Based on these observations, we propose a 
reverse-thresholdim scheme to deal with the non-stationary case. As shown in Fig 15, when the non-stationary clutter 
mean is blow the target mean, we set the criteria for assigning a detection as the object's feature value is above the 
threshold, while when the clutter mean changed to above the target mean, we set the criteria for assigning a detection as 
the object's feature value is below the threshold. This technique needs the real time measurements of the changing mean 
of a non-stationary process. This task may be conducted by using a temporal moving widow or the Wiener and/or 
Kalman filtering techniques. 

8. SUMMARY 

1) . Both the pre- and post-detection temporal integrations considerably enhance target detection by integrating only 
3-5 time frames (tested by real sensor noise as well as computer generated noise). 

2) . The newly developed pre-detection temporal integration techniques (Additive, Multiplicative, or MIN Fusion) 
perform a little better than the traditional post-detection temporal integration technique (Persistency Test). Detection 
results can be further enhanced by combining both the pre- and post-detection temporal integrations. 



6 Presume that the noise in the frames is temporally un-correlated. 

7 Note: for the traditional CFAR scheme, the threshold itself is changing (floating) from frame to frame to keep a constant false-alarm 



rate. 
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3). The techniques developed in this study can be further used for target recognition such as the ATR using matched 
fihenng^njauon W^*^ ^ ^ ^ ^ ^ roads, and buildings, etc.) has been studied. The 
sensor clutter noise at most (> 95%) of the sensor pixels is near stationary and un-correlated between pixels as well as 
(almost) un-correlated across time frames. . , 

5) . The noise at a few pixels (looking at the grass near the road edge) has shown non-stationary properties (with 

increasing or decreasing mean across time). 

6) Based on observations over the real IR sensor clutter noise, we proposed two advanced ^holding techniques: 
the double-thresholding and the reverse-thresholding. They can outperform the traditional CFAR single-thresholding 

"T^^SS broader pdf (probability density function) than the tarcet, it 
would be important to further investigate if the broad clutter pdf is caused by non-stationary noise with a t.me-vanant 
mean or is caused by a mix of different clutter types with different stationary means. Then we can accord.ngly select 
different detection techniques such as the newly proposed double-thresholding or reverse-thresholding schemes 
discussed in the previous section j:«™„» ,„»o.h»r 

8) It's critical to further investigate and understand the non-stationary noise property under different weather 
conditions and different background scenes and textures such as grass, trees, sand, and water surfaces, etc. 
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4358-0113P Provisional Application Attachment 02 
1. INTRODUCTION 

For an IR (infrared) sensor, the raw digital images coming out from the FPA (focal plane array) A/D converter 
contain strong non-uniformity/fixed pattern noise (FPN) as well as permanent and blinking dead pixels. Before 
performing the target detection and tracking functions, these raw images are processed by a CWF (chopper-wheel-free) 
MBPF NUC (Measurement-Based-Parametric-Fitting Non-Uniformity Correction) system to replace the dead pixels and 
to remove or reduce the FPN, as shown in Figure 1 . The input to MBPF NUC is RIM^ (the raw image), where 1 *i.j< 
256, and the output is CM^ the corrected image. It is important to note that as shown in Figure 1 the IT (integration 
time) for the FPA input capacitors is a critical parameter to control the sensor's sensitivity and temperature DR (dynamic 
range). 

From the results of our FPN measurement, the STD (standard deviation) of FPN from a raw uncorrected image 
can be as high as 300-400 counts. This high count FPN will severely reduce the sensor's sensitivity (we would like to 
detect a weak target as low as a couple of counts) and hamper the target tracking and/or ATR functions because of the 
high counts FPN artifocts. Therefore, the major purpose of the NUC system is to reduce FPN for early target detection, 
and the secondary purpose is to reduce FPN artifacts for reliable target tracking and ATR. 




Figure 1 FPA & NUC System 



The CWF MBPF NUC system we have developed contains three functions: I) Permanent and blinking Dead 
Pixel Replacement (PBDPR), 2) Temperature Dynamic Range Management (DRM), and 3) FPN Estimation and 
Removal (FPN E&R). 

The PBDPR function is to replace the permanent and blinking (deviant) dead pixels using some global and/or 
local thresholding techniques. The input to this function is RIM, Jy and the output of this function is DPRy. 

The DRM function is designed for the system to optimally satisfy the requirement of the temperature dynamic 
range as well as the system sensitivity (detectivity) requirements. Different ITs (integration times) and electronic gains 
are assigned to different temperature sub-dynamic ranges (sub-DRs). The input to the DRM function is DPR& and the 
output of the DRM function is DRM U . 

The FPN E&R function is to estimate and reduce the FPN based on laboratory measurements. Piece-wise (2 
pieces) or one-piece linear curve fittings are used for different sub-DRs to estimate the FPN at different temperatures and 
ITs (integration times). The input to the FPN E&R function is DRM^ and the output of the FPN E&R function is C1M U . 
Figure 2 shows the algorithm flow diagram for the MBPF NUC system whh the three functions. 
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Functions: 



Global & Local 
Thresholding; 
Temporal variance 
Test, etc. 



For Temp > 70 deg C, 
IT = 2.5 ms, GE = low. 
For Temp: 35-70 deg C, 
IT =5.2 ms, Ge = low. 
For temp 0-35 deg C, 
IT = 9 ms, Ge = low. 
For temp -33-0 deg C, 
IT = 16.6 ms, Ge = low. 



Using piece-wise (2 pieces) 
Linear curve fitting 
for the middle and high 
Sub-DRs; 

Using one-piece linear curve 
fitting for the low and very 
high sub-DRs. 



1) 

2) 



3) 



Figure 2 Algorithm Flow Diagram of MBPF NUC 

As shown in Figure 2, the DRM is a subsystem in a NUC system. The DRM, like the automatic gain control in 
a radio system, adaptively adjusts the IT (integration time) of the FPA to meet the requirements of both the sensor 
sensitivity and operating temperature dynamic range. The DRM also handles the pixel-saturation problem by reducing 
the IT. In the traditional NUC design (the prior art) using a chopper-wheel (or blurringMefbrmable lenses), the DRM 
frequently switches the ITs to keep the pixel intensity at the middle-intensity -counts (the half quantum well) to avoid the 
pixel starvation and saturation situations. There are three problems with this design: 

.The frequent changes of IT will vary the sensor sensitivity and also need more processing power. 
In many applications of IR measurements, we want to know the objects' radiant intensity from the 
measured object irradiance (for example, in the Missile Defense Program the radiant intensity is a 
critical physical feature for target discrimination and classification). However, the calibration for 
estimating the radiant intensity is related to each operating IT. Therefore, in this case we would like 
to use a few operating ITs instead of changing ITs frequently with many different values. 
If the target pixels are saturated then it is okay to reduce the IT, however, if the saturation pixels are 
from some uninteresting objects, clutters, or from some burning CM (counter-measurement) 
objects, and if the target signature is still weak, we would not want to reduce the IT. 

In general, the CWF NUC prefers to use a small number of operating ITs so that it does not require a large 
number of coefficients for estimating the FPN. In this DRM design, we pre-select a few ITs (four ITs in this design) 
depends on the requirements for sensitivity and dynamic range, and the DRM will smartly and adaptively choose the 
appropriate IT depending on the background temperatures and pixel-saturation situations. Based on our lab 
measurements, four integration times (2.5, 5.2, 9, and 16.6 ms) have been selected for the DRM to meet the requirements 
for both the seeker sensitivity and temperature dynamic range. The basic and advanced logic functions have been 
developed for the DRM. 

Wc first decide the three IT-switching temperatures (0, 35, and 70 deg C). The decision is based on several facts 
that (i) longer IT leads to lower NEDT, shorter DR, and more image blurring caused by motion; (ii) preferring to operate 
at the linear region of the sub-DR of each IT; and (hi) the FPN-counts relationship will be more linear for a shorter sub- 
DR. The basic logics manage to select an appropriate IT based on the background temperatures and the pixel-saturation 
situation (SAT = 0, or 1). IT (=16.6ms) operates for the sub-DR from -33-0 deg C; IT (=9ms) for the sub-DR from 
0-35 deg C; IT (=5.2ms) for the sub-DR from 35-70 deg C, and IT (=2.5ms) for the situation that the target temperature 
is equal to or above 70 deg C 

The advanced logic functions in our DRM design have also been developed. The advanced logics, using some 
IT-prediction functions and using some feedback information from the outputs of Tracking and ATR (automatic target 
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recognition) functions, are designed in preventing the IT switching back and forth, and are useful in detecting and 
tracking weak targets in the counter-measurement (hot burning objects) situations. Here are some more detailed 
description for the advanced logics: 

(a). If the saturated pixels are from a target, the DRM will switch the IT to a shorter one to reduce the system 

sensitivity saturated pixe | s ^ from a clutter, or an uninteresting object, or a hot CM object, the DRM will not 

switch *^ T ^ egam ^ ^ IT _p redic t at which the target pixels will be saturated. If the IT to be switched to is 
larger than IT_predict, then the DRM will not switch the IT. 

There is an additional advantage of our DRM design over the prior art. Since the DRM operates at several pre-, 
selected ITs we know exactly the sensor's sensitivity and pixel saturation temperatures under different background 
temperatures based on our lab measurements. Therefore, we can estimate the seeker performance for target detection, 
recognition, and identification under different realistic conditions. The DRM's performances for target detection, 
recognition, and identification under different weather conditions have been estimated using the NVtherm tool from 
NVL. Several realistic examples have been included to show how well the DRM works. 

2. MEASUREMENT RESULTS, REQUIREMENTS, AND PERFORMANCE 

CONSIDERATIONS 

A CE (Cincinnati Electronics) InSb FPA's sensitivity has been estimated for ITs (integration times) ranging from 
52ms to 16.6ms under different background temperatures from laboratory measurements. Based on the CE camera 
measurement data, the FPA NEDTs (noise equivalent delta temperatures) are calculated as shown in Figure 3. We then 
calculate the NEDTs for our seeker design based on the optical settings of our desgn and the opticals ettings of the Cb 
camera The NEDTs for our seeker design range from lOmk to 46mk spanning a temperature DR (dynamic range) from 
-30 to 70 deg C. Therefore, if we use ITs between 5.2ms and 16.6ms, we can meet both the requirements for sensitivity 
and DR. 




Figure 3 NEDTs for the CE Camera 

H Based on our laboratory measurements of the CENC-256 Camera: 

For IT = 52ms, we know the dynamic range DR = 70 °C (from 0 °C to 70 °C) based on the CE camera manual and 
also our laboratory measurements. 

That is, for all the ITs <5.2 ms y the requirement for T max can be satisfied. 

2) FPA Sensitivity (NEDT): _ . .... 

From Figure 3 for IT = 5.2ms, based on our laboratory measurements, we obtained that NEDT - 35.5mk (mill 
kelvin) at 22.5 °C & F - 4 (F is the lens f-number, F/4 is the number for the CE camera), and the optical 
transmittance Tau = 0.88. 
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For the same IT (= 5.2 ms), if we change the f-number F from 4 to 2.45, and change the Tau from 0.88 to 0.5, the 
NEDT will be reduced by a factor of (4/2.45) 2 *0.5/0.88 (-= 1.56), i.e., NEDT = 22.8mk at 22.5 °C & F - 2.45 & 
Tau = 0.5. 

That is, for all the ITs 25.2 ms, the NEDT can be satisfied. 

3) For InSb FPAs, NEDT increases significantly at very low temperatures (e.g., -30 °C). That is, the FPA sensitivity is 
low at low temperatures. NEDT = 46mk at -28 deg C & (F = 2.45, Tau = 0.5). 

3. OVERALL DESIGN CONSIDERATIONS AND DESCRIPTION 

The design dilemma is mat if we increase the system gain to meet the sensitivity requirement, the DR 
requirement may not be met, and vice versa. In this design, we divide the DR into three sub-DRs, and assign different 
ITs and electronic gains to the three specific sub-DRs. The purpose is to obtain as high as possible sensitivity for each 
sub-DRs, and still to be able to meet the minimum and maximum background temperature requirements. In addition, we 
select an IT = 2.5 ms when a very hot target is over 70 °C to avoid the target saturation. This IT (= 2.5 ros) is also 
selected when the missile has established Unstable tracking and/or ATR, or when the missile is in the End-Game mode. 
In some NUC systems, the NUC is turned off during the End-Game mode since the system sensitivity will not be a 
problem at this mode. However, as discussed above, the secondary purpose of NUC is to reduce the high counts FPN 
artifacts to enhance tracking and ATR. In this design, by selecting a small IT, we can avoid the strong target saturation, 
and in the meanwhile, can still continue to operate the NUC function to suppress the FPN artifacts. 

The selected ITs for the four sub-DRs are listed below: 

t) For Target Temperatures > 70 deg (saturated target intensities) & Background Temperature: -10 - > 70 °C, 
IT - 2.5 ms. 

2) For Background Temperature: 35 - 70 °C, 
IT = 5.2 ms. 

3) For Background Temperature: 0-35 °C, 
JT = 9ms. 

4) For Background Temperature: -33 ~ 0 °C, 
IT = 16.6 ms. 

4. DRM ALGORITHM DESCRIPTION 

Four sub-DRs are selected using temperature units. Based on laboratory measurements for each individual FPA, 
each specific background temperature is related to a specific image mean (averaged) value in the unit of COUNTS. In 
this DRM algorithm, we measure and use the MIC (mean image count) to find out the operating sub-DR. For example, 
M1C0J6 1 means the MIC at 0 °C & IT = 16.6 ms, and MICJ)_9 means the MIC at 0 °C & IT = 9 ms, etc. The MIC 
can be calculated from the center 64 x 64 elements of the FPA. 

For adaptively switching sub-DRs to guarantee the target signatures to be within the linear dynamic range, it 
may be necessary to know whether there are saturated pixels caused by very hot targets or objects. 

SAT = /, if there are M pixels with values larger man 4050, 
SAT = 0, otherwise 2 . 



1 This value may be determined by laboratory measurements or factory calibrations 

2 Under some situations, the saturated pixels may be from some uninterested objects or clutters and the signatures of a target may still 
be very weak. In these situations, we would like to keep the seeker in the high sensitivity mode, and thus a smarter algorithm is 
needed to know whether the saturation pixels are from a target or not 
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M is a selectable parameter (e.g., 2 - 3% of the center 64 x 64 pixels). The saturation pixels can be estimated by a fast 
histogram method implementing in an FPGA 3 . An example using a MATLAB code to calculate the histogram of center 
64 x 64 pixels is presented in the Appendix section. 

There are some other conditions: 

Tracking = /, if reliable tracking acquired, 
Tracking = 0, otherwise. 



ATR = 7, if reliable ATR acquired, 
ATR = 0, otherwise. 

A saturation-related-integration-time-prediction (SRITP) function is included The SRITP function Is used 
when SAT = 0, and the DRM is going to change the current IT to a higher IT. This function predicts the IT that increases 
the system gain so that there are M pixels with values larger than 4050 (i.e., SAT = 1). The DRM can then make an . 
appropriate decision whether to change the IT or not by comparing the predicted IT with the IT to be changed to. This 
setting allows the DRM to work under a more steady state without switching the IT back and forth. An example using a 
MATLAB code to predict the IT is presented in the Appendix section. 

In general, when the seeker starts to look, the system is still not quite under the stationary status and the targets 
may be quite far away from the seeker. Therefore, it is reasonable to start the NUC operation with the middle 
temperature sub-DR (we set the first 5 initial image frames at this mode). The detailed DRM algorithm is described 
below using a pseudo program code: 

ITjiumber(l:4) - [2.5, 5.2, 9, 16.6]; 

% Initialization, 

SubDR = 3; % I for IT=2.5 ms. 2 for IT « 5.2 ms. 3 for 1T=9 ms. and 4 for IT=16.6 ms 

For 1 <, frame ji umber <, 5 

Set IT = 9 ms & GAIN_SEL = 0; % 1 for high gain and 0 for low gain 

Low bound = MIC0__9; 

high_bound = MIC_35_9; 
End 

% Start the 6th frame. 
For 5 < frame jiumber < n, 

IfSAT = 0, 

IT j>rtdkt = SRlTP(CurrentFrame, IT_number(SubDR), M) * 
End 

If MIC_cf < lowjjound - B 5 , and SubDR * 4, % MIC cf is the current frame MIC 
SubDR = SubDR +1; 

If SAT = I, and (Tracking = I or ATR = I), 6 



3 During the NUC process, for the tracking mode, we can use the center 64 x 64 elements to compute the histogram. However, for the 
ATR mode, it may be advantageous to use the whole 256 x 256 elements. To reduce the computational intensity under this condition, 
we can under-sample the 256 x 256 image using every 4* or 8* pixel. 

4 This is the saturation related integration time prediction (SRITP) function. 

5 B is an adjustable buffering integer number, e.g., setting at 100. It prevents the integration times switch back and forth when the 
temperature during the flight is close to the two IT-switching temperatures (0°C and 35°C). 
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SubDR = SubDR -1; 

Else IfSAT = 0, and IT_iiumber(SubDR)> ITj>redict, and (Tracking = 1 or ATR= I), 7 
SubDR = SubDR - I; 

End 

Else If MIC_cf > highbound, and SubDR * 1, 
SubDR = SubDR - 1; 

Else if MlC_cf <high_bound, and SAT = 1 , and (Tracking = 1 or ATR = l), 8 and SubDR * 1 , 
SubDR = SubDR- 1; 

End 

If SubDR = 4, sel IT=16.6ms; high bound = MIC J>J 6; end 
If SubDR = 3, set IT=9ms; Jow bound = MIC0_9; highbound = M1C_35_9; end 
If SubDR = 2, set IT=5.2ms; low_bound - M1C_35_5; high_bound = MIC_70_5; end 
If SubDR = I, set IT=2.5ms; lowjxwnd = MIC_70_2; end 

End 

The logics in the DRM algorithms from the above pseudo program code arc illustrated using a block diagram in 
Figure 4. The logics are appropriately set to guarantee that the IT does not change more than one step up or down during 
one time frame. The two green boxes *If MIC < low bound* and 'If MIC_cf > high bound* are the basic logics 
functions, which determine if the DRM stays at the current IT or switches to a lower or higher IT. The other three pink 
boxes are the advanced functions dealing with the IT prediction and pixel saturation situations. These advanced 
functions use the information fed back from the tracking and/or ATR functions. If the ATR or tracking are not available 
during the early tower test, just simply set ATR = 0, or tracking = 0. If both ATR and tracking are not available during 
the test, just set both ATR = 0 and tracking = 0. The DRM will use the basic logics functions. Therefore, in essence, the 
DRM with the basic logic functions is a feedforward system as already shown in Figure 2, while the DRM with the 
advanced logic functions is a feedback system as illustrated in Figure 5. 

In summary, the DRM divides the required DR (-33 ~ 63 °C) into three sub-DRs by selecting three different ITs 
(5.2, 9, and 1 6.6 ms). With such an arrangement, the NUC system can meet both requirements for operating temperature 
dynamic range and system sensitivity. In addition, uhe DRM also select an IT (= 2.5 ms) to avoid pixel saturation for a 
very hot target (or a close target). With the additional information from conditions such as SAT, Tracking, and ATR, the 
DRM can adaptively select the appropriate ITs to guarantee that the target intensity is within the linear dynamic range, 
even for a very hot target. The maximum time for switching from the lowest temperature sub-DR to the highest 
temperature sub-DR (or vice versa) by the DRM is a sampling duration of three frames. 



6 For SAT - 1, if Tracking = 1 or ATR = I, the DRM stays in the current sub-DR without changing the IT. However, if both Tracking 
= 0 and ATR - 0, then it is more likely that the saturated pixels are related to some uninterested objects, counter-measurement burning 
objects, or clutters. Therefore, the DRM switches to a higher IT for a higher sensitivity despite the pixel saturation situation. 

7 With this logic, the DRM predicts that if the IT changes to the next higher one then the next frame will be saturated, and that the IT 
will be changed back to the current IT at that frame. With this knowledge, the DRAM decides to stay in the current IT. This setting 
allows the DRM to work under a more steady state without flip-flopping the IT back and forth. 

8 On the other hand, if SAT = 1, but both Tracking = 0 and ATR = 0, then it is more likely that the saturated pixels are related to some 
uninterested objects, counter-measurement burning objects, or clutters. Therefore, the DRM stays in the higher sensitivity sub-DR to 
search for the weak target. 
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Figure 4 Logics of the DRM 
Green Boxes: the basic logic functions; Pink Boxes: the advanced logic 



n R M with the basic logics: a feedforward system; 
•nPM tv irh the advanced logics: a feedback system 





From the Outputs 
of Tracking and 
ATR 



Figure 5 DRM with Advance Logic Functions 
5- PERFORMANCE EVALUATION 

well the DRM works. 

NVTherm is a PC based tool that can model staring thermal imagers with mid and far IR spectral bands (3 to 12 
mlcromen^i) ^cTU* camera or seeker parameters and other optical des.gn parameters to the 
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NVTherm tool to get the ranges for detection, recognition, and identification under different weather conditions and 
atmospheric transmission coefficients. For example, some critical parameters we can put into are: f7n umber, optical 
transmittance Tau, target size, Delta T, background contrast, FOV, integration times, D*, sensor type, sensor size, and 
weather conditions (Beer's law or Modtran mode!), etc. 

Target : a tank with a side view of 2.3 x 6.4 meters 
Delta T = 1 .2 5 deg C 

Range vs. NEDT 
Tau = 0 .8 S 




8 a ? S 8 
NEDT 



Figure 6 Performance of Target Detection 



Range vs. NEDT 
Tau = 0.85 




NEDT 



Figure 7 Performance of Target Recognition 

In our NVTherm runs, we followed the target and weather models used by CE (Cincinnati Electronics), as indicated in 
the manual of the CE camera (with an InSb middle wavelength IR FPA). The good and limited atmospheric extinction 
coefficients are taken to be y - 02/km and y = 1/km respectively. Therefore, for good weather the atmospheric 
transmittance Tau = exp(-0.2) = 0.82, and for limited weather Tau - exp(-l) = 0.368. The standard target model for a 
tank's side view is 2.3 x 6.6 meters, and the tank target Delta T is 1.25 deg C. The N50 (50% probability target 
detection) criteria are 0.75 cycles for detection, 3 cycles for recognition, and 6 cycles for identification. 

The performances for target detection, recognition, and identification as a function of sensor sensitivity (NEDT) 
under Good Weather Condition are shown in Figure 6, 7, and 8, respectively. Both N50 and N90 (90% probability target 
detection) results are presented. The N90 criteria wouid be more appropriate for our missile applications. The results 
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show that our DRM is working quite well under the good weather. For example, for a low sensitivity at NEDT » 50mk, 
we can start to detect the target at a range beyond 12 km, to recognize (tracking and ATR) the target at a range beyond 
3.5 km. 



Range vs. NEDT 
Tau » 0.85 




8 8 8 a ? 3 8 

NEDT 



Figure 8 Performance of Target Identification 




Figure 9 Performance of Target Detection by Beer's Law using SNR Criteria 

The major reason we used the NVTherm for performance analysis is that it incorporates both the factors of system 
sensitivity and resolution. However, the N50 and N90 detection using Johnson* s criteria are based on human eye's 
optical property. On the other hand, the missiles 1 detection generally use the CFAR (constant-false-alarm) thresholding 
technique, and thus the SNR criteria would be more appropriate for target detection in missile applications when 
operating within the range distance allowed by the optical resolution (i.e., for the longest operating range distance, the 
system MRT is stilt smaller or equal to 10 mk). The performance predictions for target detection and recognition by 
Beer's law using the SNR criteria are plotted in Figure 9, and these results are comparable to the results obtained by 
NVTherm shown in Figures 6 and 7. 
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For a specific sensor sensitivity (NEDT = 15~17mk), the performances predicted by the NVTherm for 
target detection, recognition, and identification under good and limited weather conditions are compared and the results 
are shown in Figure 10. It is seen that the bad weather can severely worsen the sensor performance. The N50 results for 
the CE camera are from the CE camera manual They are plotted here to compare with the predicted N50 and N90 
results of our optical settings. 



NEDT = 15-17 mk 




Figure 10 Performances under Good and Limited Weather Conditions 



5.1 Low-temp Sub-DR Example 

For a background temp -33 < Tb < 0 deg C, and for any target with temp 9 Tt < 30 deg C (the initial 5 time 
frames are set at IT = 9ms): the DRM will switch the IT from 9ms to 16.6ms at the 7th frame, and stay there until the 
missile hits the target. 

In more detail, if Tb = -2.5 deg C, and the target is a tank with Tt = -1 .25 deg C, then Delta T- L25 deg C, and NEDT = 
20 mk. From Figures 6, 7, and 8, under a good weather condition, we can start to detection the target at about 14 km 
away, to recognize the target at about 4.1 km, and to identify the target at about 2. 1 km. 

5.2 Mid-temp Sub-DR Example 

For a background temp 0 < Tb < 35 deg C, and for any target with temp Tt < 50 deg C (the initial 5 time 
frames are set at IT = 9ms): the DRM will set IT = 9ms at the 1 st frame, and stay there until the missile hits the target 



9 The temperatures Tb and Tt here are the apparent temperatures instead of the real temperatures. Please see Section 5.3 for the 
relationship between the apparent and the real temperatures. 
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In more detail, if Tb = 25 deg C, and the target is a tank with Tt = 2625 deg C, then Delta T = 1 .25 deg C, 
and NEDT = 15 mk. From Figure 10, under a good weather condition, we can start to detection the target at about 14.5 
km away, to recognize the target at about 4.15 km, and to identify the target at about 2.1 km. However, under a limited 
weather condition, we can start to detection the target only at about 4.1 8km away, to recognize the target at about 2.25 
km, and to identify the target at about 1 .32 km. 

5-3 Target Pixel-Saturation Examples 

Based on the Beer-Lambert law: 

Tji^TjrTau*, (0 

where T_a is the apparent target temperature at different ranges; T_r is the real target temperature; R is the range. Then 
we have: 

R = [logfTji) - log(Tj)] / log(Tau) (2) 

For a good weather (Tau = 0.85), if Tjr « 65 deg C, then T__a = 50 deg C at a range 
R = 1.61km, and Tji = 30 deg C when R = 4.76km. 

For a limited weather {Tau = 0.35), if Tj - 65 deg C, then Tji = 50 deg C at a range 
R = 0.25km, and Tjx = 30 deg C when R = 0.74km. 

For a scene with the background temp Tb = -2.5 deg C and the target real temp Tr - 65 deg C, under the good weather 
condition [Tau = 0.85): 

The DRM will switch the IT from 9ms to 16.6ms at the 7th frame, and switch back to 9ms at R « 4.76km, and further 
switch to IT = 5.2ms at R = 1 .6 1km. Then stay there until the missile hits the target. 

5.4 CM (Burning Objects) Examples (CFT2 Data) 

The CM (counter-measurement) example is from our recent captive-flight-test data. As shown in Figure II, the pixels 
of the two burning objects are saturated at counts about 4075-4095 (12 bits images). That is, SAT = I. However, the 
target signature is still quite weak at ranges longer than 2km. In this situation, we do not want to switch the IT to a 
shorter one, although SAT = 1. 

For example, suppose Tb = 25 deg C, and Tt = 26.25 deg C, and the temperature of the two burning objects are above 
100 deg C. Based on the DRM's advanced logic function: if tracking = 0 and ATR = 0, the DRM will stay at IT = 9ms 
with a high sensitivity (NEDT = 15mk) to detect and/or track the weak target. 

6. SUMMARY 

1) . A CE InSb FPA's sensitivity has been measured and estimated. The NEDT ranges from 10 to 50mk spanning a 
temperature dynamic range from -30 to 70 deg C. 

2) . Four integration times (2.5, 5.2, 9, and 16.6 ms) have been selected for the DRM to meet the requirements for both 
the seeker sensitivity and temperature dynamic range. 

3) . The DRM's performances for target detection, recognition, and identification under different weather conditions have 
been estimated using the NVtherm tool. The results show that the performances are quite satisfactory. 

4) . The advanced logic functions developed for the DRM are helpful in preventing the IT switching back and forth and 
are useful in detecting and tracking weak targets in the counter-measurement (hot burning objects) situations. 
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Figure 11 Counter-Measurement (Burning Object) Example from Captive Flight Test Data 
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4358-01 13 Provisional 

1 CLAIMS 

2 LA method to identify a potential target from image data 

3 representing a scene, comprising: 

4 performing at least one of a pre-detection temporal fusion and a pre- 

5 detection spatial fusion of the image data; and 

6 thresholding the image data after performing the step of performing. 
7 

8 2. The method of claim 1 , wherein: 

9 performing the pre-detection temporal fusion comprises temporally 

10 integrating the image data from a particular sensor across a plurality of time 

1 1 frames; and 

12 performing the pre-detection spatial fusion comprises fusing the image 

13 data from a plurality of sensors across a single time frame. 
14 

15 3. The method of claim 1, wherein the pre-detection temporal fusion 

16 and the pre-detection spatial fusion includes at least one of an additive fusion, 

17 multiplicative fusion, MIN fusion, and MAX fusion. 
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1 4. The method of claim 1, wherein the thresholding includes at least 

2 one of: 

3 a double-thresholding wherein an upper and a lower bound thresholds 

4 are set to identify the potential target; and 

5 a reverse-thresholding wherein a potential target identification level is set 

6 to be below a particular threshold. 
7 

8 5. A method to identify a potential target from image data of a scene, 

9 comprising: 

10 thresholding the image data from across multiple time frames or 

1 1 multiple sensor platforms; and 

12 fusing the image data after performing the thresholding. 
13 

14 6. The method of claim 5, wherein the thresholding includes at least 

15 one of: 

16 double-thresholding the image data, wherein an upper and a lower 

17 bound threshold are set to identify the potential target; and 

18 reverse-thresholding wherein a potential target identification level is set 

19 to be below a particular threshold. 
20 

21 7. The method of claims 1 or 5, wherein said step of thresholding 

22 produces thresholded image data, the method further comprising: 

23 identifying candidate targets from the thresholded image data. 
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1 

2 8. A device to identify potential targets from image data 

3 representative of a scene, comprising: 

4 a fusion module configured to perform at least one of integrating the 

5 image data across a plurality of time frames and integrating a plurality of 

6 images in a single time frame; and 

7 a threshold module configured to apply thresholding techniques on the 

8 image data. 
9 

10 9. The device of claim 8 wherein said device further includes at least 

1 1 one sensor configured to sense the scene across the plurality of time frames. 
12 

13 10. The device of claim 8, wherein fusion module is configured to 

14 perform at least one of an additive fusion, multiplicative fusion, M1N fusion, 

15 and MAX fusion. 
16 

17 11. The device of claim 8, wherein the fusion module is configured to 

18 perform at least one of a pre-detection fusion and a persistence test. 
19 

20 12. The device of claim 8, wherein the threshold module is configured 

21 to perform at least one of a double-thresholding technique and a reverse- 

22 thresholding technique. 
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4358-0113 Provisional 
ABSTRACT 

It is advantageous to identify potential targets as far away as possible. 
For a simple background scene such as a blue sky, a target may be recognized 
from a relatively long range distance. However, for some high clutter situations 
5 such as maintains and cities, the detection range is severely reduced. 
Moreover, clutter situation are often complicated. Also the background clutter 
may be non-stationary. In these types of situations, the traditional constant 
false alarm ratio (CFAR) detection technique often fails. To solve the problems 
discussed above, spatio-temporal fusion or integrating may be used to enhance 
10 potential target recognition. Also, detection schemes, such as double- 
thresholding and reverse-thresholding techniques, may be applied to further 
enhance potential target recognition. 
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